COMPARISON OF OPTIMAL TARGET SELECTION RULES 
BASED ON DETERMINISTIC AND STOCHASTIC 
LANCHESTER MODELS OF HETEROGENEOUS COMBAT 




Robert Lawrence Powers 



Monterey, California 




[1 i£»5,Ts| I'” 8 ®* fi V2TW»^ 

1 ll li ILssaa <5a=j U ^»e»jS'' 



Comparison of Optimal Target Selection Rules 
Based on Deterministic and Stochastic 
Lanchester Models of Heterogeneous Combat 



by 



Robert Lawrence Powers 



Thesis Advisor: 



J . G . Taylor 



December 1972 







App-ioucd {oh. pub tic ficJcas c; dii,t'u.butlon imtimttcd. 



Comparison of Optimal Target Selection Rules 
Based on Deterministic and Stochastic Lanchester Models of 

Heterogeneous Combat 

by 



Robert Lawrence ^Powers 
Lieutenant Commander, United States Navy 
B.S., United States Naval Academy, 1964 



Submitted in partial fulfillment of the 
requirements for the degree of 



MASTER OF SCIENCE IN OPERATIONS RESEARCH 



from the 



NAVAL POSTGRADUATE SCHOOL 
December 1972 



Library 

.Naval Postgraduate Schoc 
Monterey, California 9394 



ABSTRACT 

The simplest target selection problem in heterogeneous 
combat is the two-on-one battle. For a prescribed duration 
battle, deterministic and stochastic models using 
Lanchester's square law attrition mechanism are developed. 
Solutions of these models, obtained by the application of 
optimal control theory, are given, including the complete 
solution of the deterministic model and the optimal 
target selection rule and expected pay-off for special 
cases of the stochastic model. A numerical approximation 
for the general case of the stochastic model is obtained. 
Comparison of numerical results shows that the target 
selection rule specified by the deterministic models 
differs from the one given by the stochastic model. 
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I. INTRODUCTION 



One of the chief functions of mathematical models 
of combat is to provide insight for the development of 
tactics. In general, the assumptions needed to make the 
model tractable are such that the results cannot be direct- 
ly applied. Frequently, however, the results reveal a 
structure which is obscured by the complexity of the real 
world problem. 

There are a number of different approaches to combat 
attrition modeling, each with its own advantages and dis- 
advantages. Generally speaking, greater reality is intro- 
duced into such models at the expense of increased 
complexity . 

A factor to be considered is whether the results are 
influenced by the approach used. It may be that for a 
particular type of problem, a complex, difficult approach 
yields nothing that could not be gained from a simpler 
model . 

One of the first mathematical models of combat was 
presented by Rear Admiral Bradley A. Fiske, USN, in 
1905 [1]. Fiske considered an engagement between two 
fleets. He assigned a numerical value representing strength 
to each fleet and assumed that the damage a fleet could 
inflict was proportional to its strength. In a series 
of tables, he showed that the value of the weaker fleet 
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in relation to the stronger fleet decreased at an acceler- 
ating rate as the battle progressed. Fiske's logical 
development and conclusions are the same as those later 
formalized by Lanchester. 

Pioneering work on combat attrition modelling was done 
by F. W. Lanchester [2] in the early nineteen hundreds. 

He postulated that the combat attrition process could be 
described by a system of ordinary differential equations. 

Lanchester considered two types of combat: individual 
combat as represented by ancient combat; and modern com- 
bat characterized by the introduction of weapons with 
large firepower. 

Letting x(t) and y(t) represent the opposing force 
levels, in the first case, the attrition depends on both 
force levels and is given by 



^ = -ax(t )y(t ) 


(1) 


= -bx(t)y(t) 


(2) 



where a and b are called Lanchester attrition-rate coeffi- 
cients. Dividing Eq. 1 by Eq. 2 and integrating yields 
the force level equation: 

b(x(0) - x(t)) = a (y ( 0 ) - y(t)) 

which is known as the Lanchester linear law. If x is to 
win (x(t) > 0, y(t) = 0) , then the initial force levels 
must satisfy 
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x(0) > £ y ( 0 ) 

In the second case, the attrition depends only on the 
opposing force level and is given by 

at " - ay(t) 

- -bx(t) 

Prom this system of equations, Lanchester 1 s square law, 

b (x 2 ( 0 ) - x 2 ( t ) ) = a (y 2 (0) - y 2 (t)) 
is derived with the condition for x to win that 

x 2 (0) > £ y 2 (0) . 

Since Lanchester' s work, numerous extensions, modifica- 
tions, etc., have been advanced. A survey of the work in 
this area can be found in Dolansky [ 3 U • 

An obvious drawback to the Lanchester formulations is 
that they are deterministic. Given the attrition coeffi- 
cient, the winner is determined by the initial force levels. 
Clearly, this is not true in actual combat. There are a 
great many other factors, which may be considered as random 
variables, which enter into the determination. Apparently 
the first stochastic formulation was given by Koopman [4,5]. 

Weiss [6] noted a number of deficiencies. Of particular 
significance is the fact that the Lanchester model assumes 
that all units on a side are of the same type (homogeneous 
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forces). In actual combat, there are units of many dif- 
ferent types on each side (heterogeneous forces) . 

The simplest heterogeneous force model involves a 
single force, Y, facing an opposing force composed of two 
types of units, Xj and X 2 . The attrition structure is 
described by some function of attrition coefficients and 
force levels. The Y force commander faces an allocation 
of fire problem. How should he divide his fire between 
X, and X 2 ? 

. In the following, deterministic and stochastic models 
of this problem are developed and the solution obtained 
by application of optimal control theory is given. For 
particular numerical values, the optimal allocation of 
fire and the terminal pay-off are compared. 
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Table I. Notation 



Xj(t) } x 2 (t), y(t) — State variables, number of survivors 
of Xj, X 2 , Y at time t. 

T — Specif ied 'maximum duration of the battle. 

p, q, r — Value of a survivor of the battle of X p X 2 , Y 
respectively . 

a x — Attrition coefficient, rate at which one y attrits X x . 

a 2 — Rate at which one y attrits X 2 . 

b x — Rate at which one x x attrits Y. 

b 2 — Rate at which one x 2 attrits Y. 

<|> — Decision variable, fraction of Y fire directed at 

X x , 0 < <f> < 1. 

a i b i 

R = r— 

a x p 
6 ~ a 2 q 

s = bjX x (0) + b 2 x i (0) 

-7 _ R - s 
L " R - 1 

a = R(Z 2 - 1) - Z 2 
(Z - l) 2 

B = R(Z 2 - 1) - z 2 

z 2 

a = - /— 
q / a 2 
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II. THE DETERMINISTIC MODEL 



The development of the model requires three major 
decisions. An objective function must be chosen. A stop- 
ping rule must be defined and the structure of the attrition 
mechanism selected. 

Among the Y force commander's objectives might be to 
maximize casualties inflicted, minimize the casualties 
incurred or to maximize the duration of the battle. The 
objective function used in this model is 

max ry(T) - pXj(T) - qx 2 (T) 

4 > 

that is, the Y force commander desires to allocate his 
forces to maximize the net worth of the survivors, the 
terminal pay-off. Appropriate choices of the. weighting 
factors, p, q, and r, reflect Y's feeling regarding casual- 
ties. At one extreme, p and q set equal to zero correspond 
to minimizing casualties incurred, while r set equal to 
zero essentially maximizes casualties inflicted. It can 
be shown that maximizing the duration of the battle leads 
to a different allocation policy. 

There are three ways in which the battle may end. One 
force may decide to disengage, either as a function of 
casualties incurred or after some specified time, or one 
force may be driven to zero. In the deterministic formula- 
tion, there is a one-to-one correspondence between casual- 
ties and elapsed time. 
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The following stopping rule is used in this model. 

With T specified, there are four states in which the battle 
can end: 

C, y(T) > 0 and either x^T) > 0 or x 2 (T) > 0 or both 
(time expires ) . 

C 2 y(t) = 0 while either x } (t) > 0 or x 2 (t) > 0 or 
both for 0 5 t < T (Y loses). 

C 3 .Xj(t) = x 2 (t) = 0, y(t) > 0 for 0 1 t < T (X loses) 

C„ Xj(t) = x 2 (t) = y(t) = 0 for 0 5 t < T (draw). 

States C 2 to C 4 are referred to as f ight-to-the-f inish or 
terminal control battles, while C is a prescribed duration 
battle. The model is formulated as a prescribed duration 
battle with consideration given to premature terminations 
(C 2 , C 3 , or C 4 ). If T is large enough, the premature 
terminations will determine the actual time the battle ends 
The attrition structure used is the Lanchester square 
lav,'. These choices lead to the following model. 

- px j (T) - qx 2 (T) (3) 

4>a 1 y(t) 

(l-4>) a 2 y(t) (4) 

(b j x j (t) + b 2 x 2 (t)) 

0. T specified. 



max ry(T) 
0<<t><l 



ST 



dx . 
dt 



dx, 

dF 



d£ = 

dt 



x 2 , y > 
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A number of assumptions are involved in the formulation 
and use of the model. 

Al. Each unit has perfect information regarding the 
state and location of enemy units. 

A2. Fire may be shifted instantaneously. 

A3. Fire is uniformly distributed over surviving units. 

A4 . All enemy units are within range. 

A5. The effects of successive rounds are independent. 

A6. The attrition coefficients a 13 a 2 , b 13 b 2 are 
constant throughout the battle. 

A7. Fractional casualties are allowed. 

A8. Sjbj > ^ 2 ^ 2 * Since the labelling of forces is 

arbitrary, this is a nonrestrictive assumption. 

Al through A 5 are inherent in the Lanchester square lav,' 
formulation, as developed by Barfoot [73- 

This problem was first formulated and partially solved 
by Isbell and Marlow [8], before Pontryagin announced his 
maximum principle. Using subsequent developments in modern 
optimal control theory, Taylor [9, 10] has obtained a 
complete solution to both the fixed-duration and the fight- 
to-the-f inish battle. 



12 



III. SOLUTION TO THE DETERMINISTIC MODEL 



The derivation of Taylor's solution is exceedingly 
lengthy and tedious. The results are summarized in Appen- 
dix A. This section discusses Taylor's solution technique 
and the insight into the structure of the optimal allocation 
policy provided by this solution. 

The solution procedure begins by forming the Hamiltonian 

H ( t j x i} p jL , <(>)=- {Pj(t) y + p 2 (t)(l-<j)) a 2 y 

+ p 3 (t) (bjXj + t> 2 * 2 )} (5) 



where p l5 p 2 , and p 3 are the dual variables associated with 
Xj, X 2 and Y respectively. 

By the Pontryagin maximum principle"*" , maximizing Eq . 5 
at all points in time yields the maximum of Eq. 3. Rewrit- 
ing Eq. 5 as 

H = - ( a x p j (t ) - a 2 p 2 (t)) y + a 2 p 2 (t) y 

+ p 3 (t) (bjXj + b 2 x 2 ) 



it is clear that the optimal control, is given by 



<)># = 



1 if a 2 p 2 (t ) 

« 

0 if a 2 p 2 (t) 



> ajp^t) 
< a x p x (t) 



( 6 ) 



This paper uses the American form of the maximum 
principle [11]. Pontryagin, et . al . , [12] use an equivalent 
form which differs in sign conventions. 
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It can be shown that a 2 p 2 = a x Pj cannot hold over a finite 
interval of time, hence the optimal control is to always 
concentrate fire against one target type. 

The adjoint system of differential equations for the 
dual variables is given by 

dp j 

at - = b . p = (t) 

dp 

dt - ■ b 2P s C) (7) 

dp 3 

= p i ( t )a x <t> + p 2 (t)a 2 (l-<j>) 

When the optimal value function is differentiable, the 
dual variables represent the change in the optimal value 
function caused by a change in the corresponding state 
variable. In particular, if no force has been driven to 
zero at the end of the battle, the final values of the 
dual variables are easily found. Differentiating Eq. 3 
with respect to the state variables yields 

Pj (T) = - p 

P 2 (T) = - q 

P 3 (T) = r. 

If either X x or X 2 is driven to zero during the course 
of the battle, the final values of the dual variables can 
no longer be determined as above; instead, the theory of 
state variable inequality constraints must be applied. 
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Taylor [Appendix E of 10] presents a detailed discussion 
of this theory. 

When a force is driven to zero, the optimal trajectory 
is said to enter a constraint sub-arc. The point of entry 
is called a corner. In this problem, a nonnegativity 
constraint is binding on a constrained sub-arc. One treat- 
ment, developed by Gamkrelidze [12] is to modify the 
Hamiltonian by adjoining the first time derivative of the 
constraints to the criterion functional with additional 
Lagrange multipliers. 

H = - Pj a t <j>y - p 2 a 2 (l-<f>)y - p 3 (b J x 1 + b 2 x 2 ) 

- u 1 (t)a 1 ^y - y 2 ( t ) a 2 ( 1-<J> ) y 



The y^'s are multipliers satisfying 



( = 0 x i > 0 

> 0 x. = 0 

l 



i = 1, 2 



as well as Gamkrelidze ' s condition 



dy ± 
dt“ 



< 



0 



for x. = 0 
l 



There are also corner conditions on the dual variables 
at an entrance to a constrained sub-arc. If X 1 is driven 
to zero at t , these are 

Pi (t^) = Pi (tj) + y i 

P 2 (t;) = P 2 (t+) 

P 3 (t ? = P,<<) 



15 



In addition , the Hamiltonian is continuous across the 
corner. Approaching the corner (tj), 0=1, p 3 b approaches 
zero, and y^O. After the corner (t*), 0=0, Xj=0, and 
y 2 =0. From the continuity of the Hamiltonian 

H(t7) = - Pi(t7)ajy - p 3 b 2 x 2 

= - p 2 (tj)a 2 y - P 3 b 2 x 2 = H(t*) 

therefore 

cl 3 . 

^ p 2 ( fc t) = ^ p 2 (tl) • 



On a constrained sub-arc with x,=0, the multiplier is 
determined from the condition 



dH* 

d0 



0 



which yields 

MjCt) = (a 2 p (t) - ajPj (t )) . 
01 i 



Since 



du . 
dF 



1_ 

a , 



i' dp, 



a 



2dt 



- a 



dpi 



i dt 



l 



P 3 (t) 

(a b - a b ) 

a , 2 2 ii 



using the adjoint equations (Eq. 7) it is clear that 
Gamkredlidze ’ s condition is only satisfied on the con- 
strained sub-arc x =0 if a J b 1 > a 2 b 2 since it is readily 
shown that P 3 (t) > 0. 
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A similar argument shows that Gamkrelidze ' s condition 
is satisfied on the constrained sub-arc x 2 =0 only if 
a b < a b. which contradicts A8 . Therefore, it is optimal 
to drive X 2 to zero only at the end of the battle, if at all. 

Since the values of the dual variables are known for 
t=T, it is convenient to define the backwards time variable 

T = T - t 0 £ t £ T 

and so 



dp, 

dx 



H at 

dt dT 



dp, 

dt 



i = 1, 2, 3 



(8) 



Substitution of Eq. 8 into Eq . 7 yields a system of differ- 
ential equations with known initial conditions which can 
be solved for the dual variables as a function of time. 

Once the dual variable functions are found, the optimal 
control is given by Eq. 6. The time at which fire is 
switched from Xj to X 2 is found from 

ajp x (t) = a 2 p 2 (t) . 



The trajectories of the state variables are easily 
determined. For an interval (t x , t 2 ) where <j> = l, Eq. 4 
becomes 

dx ! 

-rr- = - a x y ( t ) x j (t j ) given 



dx 2 

dt 



0 



x 2 ( t j ) given 



djZ. 

dt 



(bjXj (t) + b 2 x 2 (t)) 



y(t x ) given 



which can be solved to get for t x 5 t < t 2 



x 



! (t) = XjCtJ + a t - 



r (bjXjCtj) + b 2 x 2 ( 1 2 ) ) 



a i b i 



cosh ( /a ! b l t ) 



y(t x ) (bjX j (t ! ) + b 2 x 2 (t 1 )) 

sinh ( /a [ b t ) — -r- > 

/i^b7 a i b i J 

x 2 ( t ) = x 2 (t 1 ) (9) 



y(t) = y(tj) cosh ( /a [ b [ t ) 

(biX^ti) + b 2 x 2 (t 1 )) 

- s inh ( /a , b , t ) . 

^“b7 

Similarly, for an interval (t 3 , t 4 ) where <p =0, Eq. 4 
becomes 



dx 2 

dt~ = - a 2 y(t) 

= - (b jX, (t) + b 2 x 2 (t) 
which gives for t 3 < t 5 t 4 



x 1 (t 3 ) given 

x 2 ( 1 3 ) given 
y ( 1 3 ) given 



X, (t ) = X j ( t 3 ) 

( (b i x j ( 1 3 ) + b 2 x (t )) 

x 2 ( t ) = x 2 ( 1 3 ) + a 2 j — ^ cosh ( /a 2 b 2 t ) 



y(t 3 ) 

sinh ( /a b t) 



(b jXj ( t 3 ) + b 2 x 2 (t 3 )) 



a 2 b a 
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y(t) = y(t 3 ) cosh ( /a 2 b 2 t) 

( b ! X 1 ( t 3 ) + b 2 X 2 (t 3 ) ) , 

sinh (/a,b, t) 

* / aI b 7 



( 10 ) 



Neglecting drawn battles (terminal state C^,) the 
terminal states can be partitioned as shown in Table II. 
Using the optimal control determined by Eq. 6, the state 
equations, Eqs. 9 and 10 can be integrated in forward time 
to determine what combinations of parameters and initial 
force levels can lead to particular terminal states. 

These conditions define the domain of controllability. 

Two additional items must be checked. First, the do- 
mains of controllability must cover the entire state space. 
If not, then there exists a singular surface where the 
Pontryagin maximal principle does not apply and other 
methods must be used to determine the optimal control. 

Such a surface is not present in this problem. 

Secondly, it may be that the domains of controllability 
overlap. If this is the case, then the optimal control is 
the one leading to the terminal state with the largest 
pay-off. Unless one terminal state dominates the other, 
the region of overlap is partitioned by a dispersal surface 
into subregions from which optimal extremals lead to a 
particular terminal state. 

Dispersal surfaces are present in this case. The 
resolution of these areas can be found in Taylor [10]. As 
this paper is primarily concerned with numerical results, 
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Table II. Terminal States 



State 


Y 


Xi 


x 2 


T 1 


SI 


>0 


>0 


>0 


=T 


S2 


>0 


= 0 


>0 


=T 


S3 


>0 


>0 


= 0 


=T 


S4 


= 0 


>0 


>0 


1 A 


S5 


= 0 


= 0 


>0 


5t 


s6 


= 0 


>0 


= 0 


-T 


S7 1 


>0 


= 0 


= 0 


Eh 

VI 


S8 2 


>0 


= 0 


= 0 


<T 



T : Specified duration of the battle. 

Tjj Time battle actually ends. 



SI 


- S3 


partition 


Ci 


(fixed-duration battle) 


S4 


- S6 


partition 


C 2 


(Y loses) 


S7 


- S8 


partition 


c 3 


(X loses) 



destroyed first. 

2 

X 2 destroyed first. 

the question is resolved by computing the pay-off for each 
possible terminal state and choosing the maximum. 

Insight into the allocation policy can be gained by 
considering the interpretation of the quantities a^j, 
a 2 b 2 , UjPj and a 2 q. The quantity a 1 b 1 (a 2 b 2 ) is a strategic 
or long term measure. It can be interpreted as a measure 
of the rate at which Y destroys Xj's (X 2 's) ability to 
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destroy Y. On the other hand, a x p(a 2 q) is a tactical or 
short term measure, interpreted as the rate at which Y 
destroys X : 's (X 2 's) value. By A8, ajbj > a 2 b 2 , so there 
are two cases to consider. 

If ajP > a 2 q, the situation is relatively simple. 

Y receives more return in both the strategic and tactical 
sense by firing at X l5 hence it is never optimal to fire 
at X 2 unless Xj = 0. Accordingly, in this case, no 
extremals lead to states S3, S6, or S8. For all other 
terminal states, <j>* = l until X 1 =0 or Y=0. 

Things are more complicated if a a p < a 2 q, since Y 
obtains a better tactical return by firing at X 2 , but a 
better strategic return by firing at Xj. At the termina- 
tion of the battle, only the tactical return need be con- 
sidered, so clearly, <{>*(T) = 0 as long as there are any X 2 
survivors at T. 

However, since Y is more effective in destroying Xj's 
kill capability than X 2 's, it is in Y's best interest to 
fire at X x until there is just enough time remaining to 
destroy X 2 then shift fire. Otherwise, Y will incur need- 
less casualties. 
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IV. THE STOCHASTIC MODEL 



A. STOCHASTIC CONTROL PROBLEMS 

The addition of stochastic elements to the optimal 
control problem introduces several complications. There 
are a number of aspects which must be considered, since 
different interpretations lead to different courses of 
action. ' These are discussed at length by Dreyfus [13] • 

The state of knowledge of the underlying stochastic 
process must be considered. In the simplest case, the 
properties of the process may be known. On the other hand, 
if the nature of the process is not known, a learning 
element enters the problem. The course of action chosen 
must be refined as the process evolves and knowledge is 
gained. This is known as an adaptive control problem. 

Another important factor is the ability to observe 
the state of the system. In the simplest case, the state 
can be observed precisely at any particular point. If the 
state is only partially observable, then an estimation 
problem must also be treated. 

Since the trajectory is no longer precisely determined, 
the objective function becomes a random variable-. Among 
the alternatives for treating the objective function are: 
the maximization of an expected value; maximizing the 
probability of reaching a particular state; or minimizing 
the variance of the terminal state. 
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A significant difference from the deterministic problem 
is that the manner in which the control is applied leads 
to different results. There are three basic control poli- 
cies to be considered: open loop; feedback; and open 

loop-feedback . 

An open loop control policy represents the case in 
which the control must be specified before the process 
starts and cannot be altered. In a process evolving 
through time, the control is expressed as a function of 
time, given the initial state. 

A feedback or close loop policy assumes the ability 
to continuously observe the state of the system and modify 
the control. Mathematically, the control is expressed as 
a function of time and state, given the initial state. 

The open loop-feedback policy arises if it is impossible 
to continuously observe the system. Generally, the system 
can be observed only at particular points in time. In 
this case, an open loop control based on the initial state 
is applied to start the process. After an interval of time, 
the state is observed and another open loop control is 
applied, based on the observed state. This continues until 
the process terminates . 

It is fairly obvious that the feedback policy is the 
"best" of the three. The process can be rigidly controlled 
since, should the process start to behave in a nonoptimal 
fashion, it can be corrected immediately. In an open loop 
situation, a more delicate touch is required. There is no 
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way to correct any over-control. The open loop-feedback 
corresponds to the physical situation usually faced in the 
actual problem, as opposed to the model. 

In the following section, a feedback control will be 
found for a model where the underlying process is known 
and the states are fully observable. 

B. EXACT SOLUTION OF THE STOCHASTIC MODEL 

This model differs from the deterministic model in 
the structure of the attrition mechanism. The physical 
situation is the same as in Section II. Assumptions A1 
through A6 and A8 are retained. A7 is replaced by 

A7 ' . Only integer casualties can occur and Y can 
allocate only integer numbers of troops. 

The objective function is 

MAX E{ry(T) - px (T) - qx 2 (T)} 

<l> 

This model treats the prescribed duration battle. For 
reasons similar to those of the deterministic model, it is 
convenient to work in backwards time, x = T-t where T is 
the prescribed termination time. 

Instead of a deterministic square law attrition mecha- 
nism, the attrition is modelled as a stationery Markov 
process. The following probability statements are assumed 
for a sufficiently small increment of time Ax: 

A9 • Prob [ one Xi casualty in Ax] = a^c^Ax 

Prob [ more than one X x casualty in Ax] = o(Ax) 
Prob [ one X 2 casualty in Ax] = (l-<f>) a 2 yAx 



Prob [more than one X 2 casualty in Ax] = o(Ax) 

Prob [one Y casualty in Ax] = (bjXj + b 2 x 2 )Ax 

Prob [more than one Y casualty in Ax] = o(Ax) 

Prob [more than one casualty of any type in Ax] 

= o(Ax) 

where o(Ax) is a quantity such that lim - j^ T ^ = 0. Given 

Ax+0 

the above, it follows that 

P [nb Xj casualties in Ax] = 1 - (J^yAx + o(Ax) 

and similarly for X 2 and Y. 

The assumption that Y has perfect information about 
the state of the system at any time and the ability to 
instantaneously shift fire leads to a feedback or closed 
loop optimal control. 

To solve this model, the optimal expected value function, 
S,is defined as 

S[x,Xj(x) , x 2 (x), y ( x ) ] = E[ry ( 0 ) - px^O) - qx 2 (0)] 

given that the state at time x is Xj(x), x 2 (x) and y(x) 
and that an optimal control has been applied in the interval 

[0,x]. 

Table III lists five .events which could take place in 
an interval of length Ax, [x, x-Ax] and their probability 
of occurrence. An expression for S(x, x , x , y) is 
developed by the application of Bellman's Principle of 
Optimality [1^1] which states: 
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Table III. Transition Probabilities 



State @x 


Casualties 


Probability 


x 15 x 2 , y 


none 


1 - {<f>a x y + (l-<f>)a 2 y + bjX x 
+ b 2 x 2 } Ax 


Xj-1, x 2 , y 


one X 


<f>a x yAx 


x^ x 2 — 1 , y 


one X 2 


(l-4>)a 2 yAT 


Xj, x 2 , y-1 


one Y 


(bjXj + b 2 x 2 )Ax 


Various 


more than 1 


o ( Ax ) 



An optimal policy has the property that, whatever the 
initial state and initial decision are, the remaining 
decisions must constitute an optimal policy with regard 
to the state resulting from the first decision. 

By definition, the return from any of the states listed in 
Table III resulting from following an optimal policy to 
the end of the battle is S(t-At, x j5 x 2 , y). Therefore, 
the optimal value of 4> in the interval [x, t-At] is the 
one which maximizes the return expected, starting from 
state Xj, x 2 , y at time x. This can be expressed as 

S[x, x,, x , y] = max {[1 - {<t>a,y + (l-<f>)a y 

0 << J ><1 

+ b x + b x }Ax]S(x-Ax,x , x , y ) 

112 2 12 

+ <j>a 1 yAxS(T-Ax,x 1 -l,x 2 ,y) 



+ (l-4>)a 2 yAxS(x-Ax,x 1 ,x 2 -l,y) (11) 

+ (bjXj + b 2 x 2 ) AxS( t-Ax , x j ,x 2 ,y-l ) + o(Ax)} 
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Since S(x-Ax, x 13 x 2 , y) does not depend on the control 
variable, p , this can be rewritten as 



S(t, Xj, x 2 , y) - S ( t-At , xj, x 2 , y) 

__ 

= max { ( b x x j + b 2 x 2 ) [S ( t-At , x, , x 2 , y-1) 

o< 4 >< i 

-S ( t-At , x p x 2 , y ) ] 

+ y { 4>a 2 [S(t-At , x,-l, x 2 , y) - S(x-Ax, x 13 

+ (l-{j))a 2 [S(T-AT, Xp x 2 — 1 , y) - S(t-At, Xj 

, °(Ax) \ 

At 1 

In the limit as Ax->0, 
dS(T , Xp x 2 , y ) 

= (bjX, + b 2 x 2 )[S(T, Xp x 2 , y 

- S(t, Xj , x 2 j y)] 

+ y max {<j>(a [S(x, x -1, x , y) - S(x, x , 
0<<f>< l 

- a 2 [S(x, Xj, x 2 -l, y) - S(x, Xj , x 2 , y)]) 

+ a 2 [S(x, x , x 2 -l, y) - S(x, x x , x 2 , y)] 



x 2 3 y) ] 

> x 2 , y)] 

- 1 ) 

x 2 , y) ] 
( 12 ) 



27 



Equation 12 can also be obtained directly from Kushner's 

general result on the optimal control of a "Poisson" 

2 

process [15]. 

To determine the switching function 

w(x) = a x { S(t , x l -l i x 2 , y) - S(x , x, , x 2 , y)} 

-a 2 (S(x , Xj , x 2 -l, y) - S(x , x 1 , x 2 , y)} (13) 

is used. Equation 12 then becomes 
dS(x i x 1} x , y) 

^ = (b 1 x l + b 2 x 2 ){S( Xj x l# x 2> y-1) 

- S(t , x 1 , x 2 , y)} 

+ a 2 y{S(r , x l , x 2 -l, y) - S( T , x 2 , y)} 

+ y max { v/ ( x ) } 

Ol<j)<l 

Clearly, the optimal control is given by 

f 1 v; ( t ) > 0 

<$> *(t ) = \ 

[0 w(t ) < 0 

In the deterministic case where the optimal control 
depended on the dual variables, it was necessary to treat 
the case where a force is driven to zero separately since 



2 

This contains a typographical error. The unnumbered 
equation after Eq. 7 should read 

n n 

0 = min {k + V + £ p.g. + l c [V( x+X , t) - V(x, t)]} 
u 1 1 x 

If 1=1, this reduces to Dreyfus result [13]. 
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the boundary influenced the solution for the dual varia- 
bles. In the stochastic case, the optimal control depends 
only on the optimal expected value function. As shown 
below, the initial conditions for S are easily determined 
even when a force is driven to zero. Therefore, no special 
distinction between the fixed duration and the fight to 
the finish battles is necessary to solve for the optimal 
expected value function. 

If x=0 (i.e. at the end of the battle), then 

S(0, Xj, x 2 , y) = ry - px x - qx 2 ( 14 ) 

for all values x x , x 2 , and y. 

If y=0, there is no battle and 

S(t , Xj, x 2 , 0) = - (pXj + qx 2 ) (15) 

for all values of t, x , and x 2> 

Similarly, if x ( = x 2 = 0, there is no battle and 

S ( t , 0, 0, y) = ry (l6) 

for all values of t and y. 

If Xj = 0, then clearly, <p = 0 for all r and Eq. 12 
reduces to 

dS( t , 0, x , y) 

^ = b 2 x 2 [S(T, 0, x 2 , y-1 ) - S(t, 0, x 2 y)] 

+ ya 2 [S( x , 0, x 2 -l, y) - S( x , 0, x 2 , y)] 

with the initial condition from Eq. lM 
S(0, 0, X 2 , y) = ry - qx 2 . 
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In particular, for x 2 ~ y = 1 



dS(x, 0, 1, 1) 

^ = b 2 [S(x, 0, 1, 0) - S(x, 0, 1, 1)] 

+ a 2 [S(x, 0, 0, 1) - S (x , 0, 1, 1)] 

= b 2 [-q-S(x, 0, 1, 1)] + a 2 [r-S(x ,0,1,1) ] 



using Eqs. 15 and 16 or 
dS (x , 0, 1, 1) 

^ = - (b 2 + a 2 )S(x, 0, 1, 1) + a 2 r - b 2 q 

with S(0, 0, 1, 1) = r - q. This is an ordinary differen- 
tial equation of the form 



which is easily solved to give 



S(x, 0, 1, 1) 



'b x r - a 2 q' 
a 2 + b 2 



exp { — ( a 2 + b 2 ) x } 



+ 



a 2 r 




(17) 



Similarly, if x = 0, then <j> = 1 for all t and for 
x = y = 1, Eq. 12 reduces to 

dS(x , 1, 0, 1) 

^ = bl {-p-S( T , 1, 0, 1)} + a^r-SCx, 1,0,1)} 

= •- (b j + a j )S(x, 1, 0, 1) + a^ - b t p 



with S(0, 1, 0, 1) = r - p. The solution in this case is 



30 



S(t , 1, 0, 1) = 

a i r ~ b iP 



b i r* - a x p 



a i + b i 



exp {-(aj + b x ) t} 



a i + b i 



( 18 ) 



Letting Xj = x 2 = y = 1, Eq. 12 becomes 

1 "- - = ( b j + b 2 )[S(x, 1, 1, 0) - S ( x , 1, 1,1) 

+ a 2 [S(t , 1, 0, 1) - S(t, 1, 1, 1)] 

+ max { (f> T a [S(t , 0, 1, 1) - S(t, 1, 1, 1)] 

0<<}><1 

- a 2 [S(x , 1, 0, 1) - S(t, 1, 1, 1)])} . (19) 

The switching function determines $ . At the end of 
the battle 



w(0) = a x (r -q-r+p+q)- a 2 (r - p - r + p + q) 



= a t p - a 2 q 

so there are two cases to consider. 

1 . Case I 

a x p 1 a 2 q 

<j>*(i) = 1 for 0 < t < ij where ij is the time when Y 
switches fire from X t to X 2 , determined by 

w(t t ) = 0. 

This is consistent with the interpretation of a t p as a 
tactical measure. Certainly, at the end of the battle, 
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Y should direct his fire to get the greater return. In 
this case, Eq . 19 becomes 

~~ T '' > ~~ d i~~ 1 ~^~'~ = (b i + b 2 H s ( T > 1 » 1 > 0) - S(T ^ 



+ aj{S(T, 0 } 1, 1) - S(t 3 l, 1, l)} 



with the initial condition 



S<0, 1, 1, 1) = r - p - q. 

Substituting Eqs. 15 and 17, the solution is given by 



S(t, 1, l, 1) 



a! (b 2 r - a 2 q) 

(a; + b> - a 2 )(a 2 + b 2 ) exp { 



(a 2 +b 2 )t } 



C(bj - a 2 )(bj + b 2 ) + a jb j ]r a x p 

(a t + b j - a 2 ) (a x + bj + b 2 ) ~ (a \ + b [ + b 2 ) 

a i a 2 q 

(a t + bj - a 2 )(aj + b t + b 2 ) 



exp { - (a t + b j + 

(bj + b 2 )p 
(a x + bj + b 2 ) 



b 2 )x} 



+ 



' (a 2 + 



[ (b j + b 2 ) (a 2 
(a 2 + b 2 ) ( a 



b 2 ) ( a 3 + b j + b 2 1 

+ b 2 ) + a j b 2 ]q' 

i + b j + b 2 ) 



for 0 < t — t 

i 

Using this in Eq. 13, the switching time ij is deter- 
mined. Depending on the value of T, it may be that x,>T 
and so no switch occurs during the battle. If x^T, then 
in the interval Xj 5 x 1 x 2 where x 2 is the time of switch 
from X 2 back, to X l3 (w(x 2 ) = 0 ) , <J>*( T ) = 0 and 
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a 2 (b x r - a x p) 

S(T - X > 1 ‘ X) = • fa 2 "t - b 7' -"a,T(a ' 1 - V"b, 7 < exp{-(a 1+ b, )t) 



- exp { (a 2 + b 2 ) ( t j - t) - a j x i - b x }| 
f ^ ai a 2 r 

+ S ° (a t + b j ) ( a 2 + b 2 + b [ ) 

[(bj + b 2 ) (a x + b x ) + a 2 b 2 ]p (b x + b 2 )q 

+ (a x + b ~ ) (a 2 + F 2 + b : ) + (a 2 + b 2 + b^ 

& -p 

(a,tb 1 )(a 2 +b 2 +b l ) 

C (b 1 + b 2 )(aj + bj) + a 2 b,]p (b x + b 2 )q 

(a x + b x ) (a 2 + b 2 ~ + b x ) (a 2 + b 2 + b x ) 

where S 0 = S(t [ , 1, 1, 1). 

2. Case II 



a 2 q > ajp 

In this case, 4 >(t) = 0 for 0 ^ t 1 where t is the 
time that fire switches from X 2 to X x . For this interval 

a 2 (b i r “ a iP ) 

S(T - 11 = (a 2 + b 2 - a,)(a, + b,) exp < “ < a ! +b i > T > 

[ (b 2 - a^b, + b 2 ) + a 2 b 2 ]r 
+ ' (a 2 + b 2 -a'^ra'j + b 2 + b x ) 

a ja 2 p 

+ (a 2 + b 2 - a j ) (a 2 + b 2 + b x )" 

a 2 q ] 

" r aT ~ rv 2 - + bV) exp ^“ (a * + b ^ )T} 
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( 



a i a 2 r 



( a " +b ) (a , +b +b ) 



[(b x +b 2 ) (aj+bj ) + a 2 b 1 ] 
C a. i +b ! ) ( l a 2 +b 2 -. L b 1 ) 



(b 1 +b 2 )q 
(a 2 +b 2 +b i ) 



If Tj<T, then for t x 5 x < x 2 



S(t, 1,1,1) 



a j (b 2 r-a 2 q) 
Ta~+b j -a 2 ) (a 2 +b 2 ) 



exp {-(a 2 +b 2 )t} 



- exp 



{ (aj+bj) (x 1 -x)-a 2 T 1 -b 2 x} 



S. - 



(bj+b 2 )p 

(a 1 '+b 1 +b 2 ) (a 2 +b 2 ) + (a 1 +b 1 +b 2 ) 



a i a 2 r 



[ (b i+b 2 ) (a ? +b 2 )+ a i b 2 ]q 

(a j+b x +b 2 ) (a 2 +b 2 ) 



exp {(a 1 +b 1 +b 2 )(x 1 -x)} 



+ 1 



a i a 2 r 



(b i + b 2 )P 
(a x +b x +b 2 ) (a 2 +b 2 ) “ (a 1 +b 1 +b 2 ) 



[ (b x +b 2 ) (a 2 +b 2 ) + (a x b 2 ) ]q 
(a 1 +b~+b 2 ’J (a 2 +b 2 ) 



S Q = SCxj, 1,1,1) where x 2 is the time fire is shifted back 
to X 2 . 

With S(x, 1,1,1) known, there is no mathematical reason 
why Eq. 11 cannot be solved for S(x,2,l,l), S(x, 1,2,1), 
etc. in hopes that eventually a general solution for 
S(x,x x ,x 2 ,y) might emerge. For the stochastic model corre- 
sponding to the linear law attrition process, such a general 
solution has been obtained by Clark [16], 
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No general solution for this model is known. From 
the ■ complexity of the expression for S(x, 1,1,1), it is 
obvious that extensions will quickly become very unwieldy. 

A numerical approximation for S(x,x l3 x ,y) appears to be 
a more profitable line of approach. 

C. NUMERICAL APPROXIMATION OF THE STOCHASTIC MODEL 

In the development of the differential difference 
equation for S(x,x l3 x ,y), a suitable numerical approxima- 
tion was also developed. 

S(t+At ,x ,x ,y ) = max i (1 - Ua y + ( 1— cf> ) a y 

0<4><1 l 

+ bjXj + b 2 x 2 }Ax)S(T 3 x 1 ,x 2 ,y) + 4>a x yAxS( x , x x -1 , x 2 ,y ) 
+ (l-<}))a 2 yAxS(x 3 x 1 3 x 2 -l 3 y) 

+ (bjXj + b 2 x 2 ) AxS(x 3 Xj 3 x 2 3 y-l) j (11) 

with 4) determined by the value of the switching function 
w( x ) in Eq . 13 . 

This is essentially the Euler method of numerical 
approximation. It has the advantage that as Ax-*-0, the 
value computed above will converge to the true value of 
S(x 3 Xj ,x 2 3 y ) . 

To obtain a reasonable approximation, the value of 
the time step, Ax, must' be small. There is also an addi- 
tional restriction on Ax. In order for the probability 
statements assumed in A9 to be meaningful. Ax must be such 
that 
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a x yAx i 1 
a z yAx i 1 

(bj x x + b 2 x 2 ) At i 1 

for all values of x ( , x 2 and y considered. 

There are several special cases of Eq. 11. 

S(0,x l5 x 2 ,y) = ry - px 1 - qx 2 

S(t ,x x ,x 2 ,0) = - px j - qx 2 

S(x,0,0,y) = ry 

S(x+Ax ,0,x 2 ,y ) = S (x , 0 ,x 2 ,y ) 

+ Ax(b 2 x 2 {S(x ,0 ,x 2 ,y-l) - S(x,0,x 2 ,y) 

since <J> = 1 in this case and 

S ( x+Ax ,Xj j 0 ,y ) = S(x ) x 1 , 0 ,y ) 

+ Ax (bjXj {SC tjXj , 0,y-l) - S(x,x 1# 0,y)} 

+ a 1 y{S(x,x i -l,0,y) - SCxjXj , 0 } y) }) 

since <j> = 0 in this case. 

The above equations were programmed in Fortran IV G 
and run on an IBM 360 / 67 . The output is a table which 
gives S as a function of x for fixed values of x , x 2 , 
and y . 

In the special case of S(x, 1,1,1), the accuracy of the 
approximation was checked against the exact solution 
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developed in Section III B. The results of one such 
comparison are given in Table IV . It can be seen that 
the approximation agrees with the exact solution to the 
fourth decimal place. 

For a given force level, the approximation must first 
calculate the values of S for all combinations of smaller 
force levels. For larger force levels, this requires 
a large amount of computer time and memory. For a rela- 
tively large time step of At = .01, computation of 
S(t,5j 5,5) for 0 5 t < 50 required about four minutes and 
200 K of core. To solve the same problem for S(x, 9 , 9 , 9 ) 
would require about one hour and 500 K. Therefore, the 
numerical results from this particular approach are 
limited to relatively small force levels. 



37 



Table IV. Comparison of Exact Solution 
and Numerical Approximation 



T 

in minutes 


Exact 

S(x, 1,1,1) 


Approximation 


0 


-.2000 


-.2000 


1 


-.2018 


-.2018 


2 


-.2037 


-.2037 


3 


-.2055 


-.2055 


4 


-.2073 


-.2073 


5 


-.2091 


-.2091 


6 


-.2108 


-.2108 


7 


-.2126 


-.2126 


8 


-.2143 


-.2143 


9 


-.2161 


-.2161 


10 


-.2178 


-.2178 


11 


-.2195 


-.2195 


12 


-.2212 


-.2211 


13 


-.2229 


-.2228 


14 


-.2245 


-.2245 


15 


-.2262 


-.2261 


16 


-.2279 


-.2213 


17 


-.2295 


-.2294 


18 


-.2311 


-.2310 


19 


-.2321 


-.2326 


20 


-.2343 


-.2342 


21 


-.2358 


-.2357 


22 


-.2374 


-.2313 
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Table IV. (Continued) 



X 

in minutes 


Exact 
S(x, 1,1,1) 


Approximation 


23 


-.2390 


- .2388 


24 


-.2405 


-.2404 


25 


- . 2420 


-.2419 


26 


- . 2435 


-.2434 


27 


-.2450 


-.2449 


28 , 


-.2465 


-.2464 


29 


-.2480 


-.2479 


30 


- . 2-495 


-.2493 


31 


-.2509 


-.2508 


32 


- . 252^4 


-.2522 


33 


-.2538 


-.2537 


34 


-.2552 


-.2551 


35 


-.2566 


-.2565 


36 


-.2580 


-.2579 


37 


- • 259*1 


-.2593 


38 


-.2608 


-.2606 


39 


-.2622 


-.2620 


bo 


-.2635 


-.2633 


bi 


-.2649 


-.2647 


b2 


-.2662 


-.2660 


43 


-.2675 


-.2673 


44 


-.2688 


- . 2686 


45 


-.2701 


-.2699 


b6 


-.2714 


-.2712 
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Table IV. (Continued) 



t Exact Approximation 



in 


minutes 




S(t, 1,1,1) 






47 




-.2727 


-.2725 




48 




-.2739 


-.2737 




49 




-.2752 


-.2750 




50 




-.2764 


-.2762 


a i 


= .005 


a 2 = .003 


b j = .007 


b 2 = .001 


P 


= .15 


q = .45 


r = .40 


At = .01 
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V. COMPARISON OF RESULTS 



For the 125 combinations of force levels from 
x i = X 2 = y “ 1 up - t0 and including x 1 = x 2 = y = 5, 
calculations were carried out for each of the four sets 
of parameter values listed in Table V. A time step of 
.01 minutes was used in the numerical approximation. 
Comparisons of selected results are given in Tables VI 
through XIV. 

These tables were constructed by extracting the force 
level time history from the deterministic model. Since 
the deterministic model calculates force levels as contin- 
uous functions of time, there is a question as to exactly 
when a casualty occurs. For these tables, deterministic 
force levels are rounded up to the next integer, i.e. 

Xj = 3-001 is rounded to x, =4. With these values of 
time and force levels, the optimal expected value function 
and the value of <(>* were obtained from the appropriate 
stochastic table. All comparisons are for the case ajP<a 2 q. 

In the case a p > a 2 q, calculations were carried for 
the above force level combinations with parameter set 1. 

In both the stochastic and deterministic model, there is 
no shift of fire unless x is driven to zero. 



Table V. Parameter Values for 



Numerical Calculations 



Set 


a i 




a 2 


b 

1 


b 2 P 


q 


r 


T 


1 


.005 


• 


003 


.007 


.001 .15 


.45 


.4 


50 


2 


.005 


• 


003 


.007 


.001 .15 


.45 


.4 


100 


3 


.025 


• 


015 


.035 


.005 .75 


2.25 


2.0 


50 


4 


• 05 


• 


03 


.07 


.01 1.5 


4.5 


4.0 


50 




i 

Table VI. 


Comparison 


for x 1 (0) = x 2 


(0) = 


y (o) = 


1 










with Parameter Set 2 








in 


Time 

minutes 


Force Level S(t, 

Xj x 2 y Deterministic Stochastic 


Xj >x 2 




0 


1 


1 


' 1 


1 


1 


- 


• 3175 




49-94 


1 


1 


1 


1 


0 


- 


.2762 




56.38 


1 


1 


1 


0 


0 


- 


.2686 




100 


1 


1 


1 


0 


0 


- 


.2000 


Deterministic 


Pay- 


off -. 


4127 









Terminal State SI 
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Table VII. Comparison for Xj(0) = x 2 (0) = y(0) = 5 

with Parameter Set 2 

Time Force Level S(tjX 13 x 2> y) 



in minutes 


X ! 


X 2 


y 


Deterministic 


Stochastic 




0 


5 


5 


5 


1 


1 


-.6231 


27 


5 


5 


4 


1 


1 


-2.1729 


50 


4 


5 


4 


1 


1 


-1.6712 


55 


4 


5 


4 


1 


0 


-1.6373 


56 


4 


5 


3 


1 


0 


-2.0558 


56.38 


4 


5 


3 


0 


0 


-2.0501 


87 


4 


5 


2 


0 


0 


-2.1846 


100 


4 


5 


2 


- 


- 


-2.0500 


Deterministic 


Pay- 


off 


-2.0637 






Terminal State 


Si 
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Table VIII. Comparison for Xj(0) = x 2 (0) = 5 y(0) = 

with Parameter Set 4 



Time 

in Minutes 


Force Level 
Xj x 2 y 


<(> * 

Deterministic Stochastic 


S(t,x 1 ,x 2 


0 


k 


5 


5 


0 


1 


-16.9539 


n 


H 


5 


ll 


0 


1 


-20.5330 


7 


H 


5 


3 


0 


1 


-23.0807 


9 


H 


5 


2 


0 


0 


-25.3355 


10 




il 


2 


0 


0 


-20.2005 


13 




il 


1 


0 


0 


-20.9796 


15.51 


ll 


ll 


0 


- 


- 


— 2 ll . 0000 


Deterministic 


Pay- 


off 


-23.2951 






Terminal State 


Sk 










Table IX. 


Comparison 


for x j ( 0) = 2 


x 2 ( 0 ) = 


5 y (0) = 








with Parameter Set 


3 




Time 

in Minutes 


Force Level 
x i x 2 y 


<fr» 

Deterministic Stochastic 


S(t,Xj } x 2 


0 


2 


5 


3 


' 1 


1 


-8.9286 


13 


2 


5 


2 


1 


1 


-11.1566 


18 


1 


5 


2 


1 


1 


-9.1233 


31 


1 


5 


1 


1 


1 


-10.9620 


35.39 


1 


5 


1 


1 


0 


-10.7862 


ill. 28 


1 


5 


1 


0 


0 


-10.511115 


50 


1 


5 


1 


- 


- 


-10.0000 


Deterministic 


Pay- 


off 


-10. 91171 







Terminal State SI 



Table X. Comparison for Xj(0) = 4 x 2 (0) = 5 y(0) = 5 



with Parameter Set 2 

Time Force Level 4>* S(t,x ,x ,y) 



in Minutes 


x i 


X 2 


y 


Deterministic 


* 1 * 2 9 

Stochastic 


0 


3 


5 


4 


1 


1 


- 8.8300 


9 


3 


5 


3 


1 


1 


-11.2142 


13 


2 


5 


3 


1 


1 


-8.9147 


20 


2 


5 


2 


1 


1 


-11.0357 


34 


2 


5 


1 


1 


0 


- 12.0296 


36 


1 


5 


1 


1 


0 


- 10.8092 


41.276 


1 


5 


1 


0 


0 


-10.5445 


50 


1 


5 


1 


- 


- 


- 10.0000 


Deterministic 


Pay- 


off 


-11.6727 







Terminal State SI 

Table XI. Comparison for x ; (0) = 4 x 2 (0) = 5 y(0) = 5 

with Parameter Set 2 



Time 

in Minutes 


Force 

x i 


Level 

x 2 y 


** 

Deterministic Stochastic 


S(t > x 1 ,x 2 ,y) 


0 


4 


5 


5 


0 




1 


-8.4801 


7 


4 


5 


4 


0 




1 


-10.4644 


13 


4 


5 


3 


0 




1 


-11.7339 


19 


•4 


5 


2 


0 




0 


-12.6693 


20 


4 


4 


2 


0 




0 


- 10 . 3609 


25 


4 


4 


1 


0 




0 


-11.1032 


31.03 


4 


4 


0 


— 




— 


-13 



Deterministic Pay-off -11.6476 
Terminal State S4 



Table XII 




Comparison 

with 


for x j ( 0 ) = 4 
Parameter Set 


x 2 (0) = 

2 


^ y(o) = 5 


Time 


Force ! 


Level 


4) * 




S(t,x 1 ,x,y) 


in Minutes 


X 


i x 


y Deterministic Stochastic 




0 


4 


4 


5 


1 


1 


-6.2034 


7 


4 


4 


4 


1 


1 


-8.8186 


10 


3 


4 


4 


1 


1 


-6.225 


16 


3 


4 


3 


1 


1 


-8.655 


23 


2 


4 


3 


1 


1 


-6.188 


26 


2 


4 


2 


1 


1 


-8.475 


39 


2 


4 


1 


1 


0 


-9.517 


41.28 


2 


4 


1 


0 


0 


-9.400 


50 


2 


4 


1 


- 


- 


-8.500 


Deterministic 


Pay 


-off - 


9.2835 







Terminal State SI 
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Table XIII. 


Comparison 


for x i ( 0 ) = 3 


i! 

O 

CM 

X 


5 y ( 0 ) = 








with Parameter Set 


2 




Time 

in Minutes 


Force Level 
x i x 2 y 


9* 

Deterministic Stochastic 


S(t,Xj ,x 2 . 


• 0 


3 


5 ■ 


4 


1 


1 


- 8.1836 


9 


3 


5 


3 


1 


1 


-11.214 


13 


2 


5 


3 


1 


1 


- 8.918 


20 


2 


5 


2 


1 


1 


- 11.036 


34 


2 


5 


1 


1 


0 


- 12.030 


36 


1 


5 


1 


1 


0 


-10.809 


41.28 


1 


5 


1 


0 


0 


-10.545 


50 


1 


5 


1 


- 


- 


- 10.000 


Deterministic 


Pay- 


off 


-11.6727 






Terminal State 


Si 










Table XIV. 


Comparison 


for x t ( 0 ) = x ^ 


( 0 ) = y ( 0 ) 


= 5 with 










Parameter Set 1 






Time 

in Minutes 


Force Level 
x i x 2 y 


<j>* 

Deterministic Stochastic 


S( t jX pX^ j 


0 


5 


5 


5 


1 


1 


-1.509 


5.61 


5 


5 


5 


1 


0 


-1.446 


6.38 


5 


5 


5 


0 


0 


-1.435 


26 


5 


5 


4 


0 


0 


- 1.665 


50 


5 


5 


4 


_ 


— 


-1.400 



Deterministic Pay-off -1.5216 



Terminal State SI 



VI. CONCLUSIONS 



Due to the restrictions of the stochastic model to 
small force levels because of computer constraints, it is 
difficult to reach any general conclusions. There are, 
however, several interesting things which can be observed 
from the data. 

In the case where a 2 q>a 1 p, the models lead to target 
selection rules where the time fire is shifted differs by 
as much as seven minutes. If the deterministic rule is to 
shift fire before X x is destroyed, the stochastic rule is 
to shift fire at an earlier time. This can be explained by 
an interpretation of the model parameters similar to that 
given in Section III. 

The return obtained by firing at X 1 is mainly from 
destroying his kill capability. More Y's survive for a 
longer period of time. ' As the battle progresses, the 
utility of Y survivors in the future decreases until a 
point is reached where the return from firing at X drops 
off rapidly. Conversely, the return from firing at X 2 is 
chiefly from destroying his value. As the battle progresses, 
the return from firing at X, eventually exceeds that of 
firing at X x . In the deterministic model, this crossover 
point is known precisely. On the other hand, this cross- 
over point is not precisely known in the stochastic model. 

The use of expected values tends to force an earlier shift 
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to keep away from the area where the X return decreases 
rapidly. In other words, some of the future value of Y 
survivors is sacrificed to ensure that the higher return 
from firing at X 2 is not missed. In the case where 
a 1 p>a 2 q, this does not occur since it is always to Y’s 
advantage to fire at X l3 both in terms of value destroyed 
and future survivors . 

The large jump in the optimal expected value function 
as casualties occur is caused by the integer constraints 
on the stochastic model. This does not happen in the 
deterministic model since force levels are calculated as 
continuous functions. As t 4 T, the optimal expected value 
function approaches the deterministic pay-off. 

There are a number of areas which need additional work. 
Most obvious is the need for better numerical techniques 
for the stochastic model. One alternative is to reprogram 
the model to use tape storage rather than core. In this 
way, a series of small computer runs could be used to build 
up a file of values for smaller force levels which could 
then be accessed to compute still higher force levels. In 
this case, an investigation of the truncation errors would 
have to be made to determine the degree of accuracy in the 
figures for higher force levels. 

Another alternative is to consider a diffusion approxi- 
mation to the stochastic process. Some work in this area 
has been done by Taylor [10], leading to a parabolic partial 
differential equation which is not easily solved 
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analytically. It can be solved numerically. This approach 
appears promising since it would not be necessary to solve 
for all combinations of force levels below the desired ones 

This approach to the stochastic model yields no informa 
tion about the time history of the average force levels 
or the probability of winning. Clark [16] was able to 
obtain expressions for the average force levels in the 
linear law case. If a similar solution for the square law 
case could be obtained, more meaningful comparisons could 
be made. 

There are also a number of extensions to the determin- 
istic model. The zero-one nature of the solution is hardly 
realistic. It is a result of the square lav; attrition 
mechanism and the constant attrition coefficients. One 
area of investigation would be to make the attrition coef- 
ficients a function of the fire being received. Other 
areas of extension are the two-on-two, one-on-m, and m-on-n 
battles with various types of attrition mechanisms and 
coefficients . 

In summary, the numerical results, while inconclusive, 
do indicate a significant difference in results between 
the stochastic and deterministic modelling approaches. 
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APPENDIX A. 



Domains of Controllability 



Case I . a x p>a 2 q 



SI J 



x x (T) > 0 



x (T) > 0 



ct ) = 1 o < t < T 



y (T) > o 

Requires T < TS1 where 



TS1=H 



/a b 
v i i 



/a i b i 



> / a 1 b 1 



/a i b i 



In {y} if a 1 b 1 y 2 (0) > s : 



In 



b 2 x 2 (0) 



if a x b x y 2 (0) = s 



In {-y} if s 2 - (b 2 x 2 ( 0 ) ) 2 _ a 1 b 1 y 2 (0)<s : 



tanh 



-1 



^b j y ( 0 ) j 



if a 1 b 1 y 2 (0) < s 2 - (b 2 x 2 (0)) 



/ a jb x y 2 ( 0 ) - s 2 + (b 2 x 2 (0)) - b 2 x 2 (0) 

Y = ■ ■ 

/a7b7 y(0) - s 



S2 



x j (TS1 ) = 0 
' x 2 (T) > 0 



t y ( t ) >> o 



4>*(t) 



'l 0 i t 1 TS1 
TS1 < t 1 T 

\ 
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Requires a 1 b 1 y 2 (0) > s 2 -- (b 2 x 2 (0)) 



TS1 < T 



S3 



No extremals lead to this state 



S4 



x x (T) 0 



x, (T) = 0 



y(T) > o 



fx j (Tl) > 0 



x 2 (T1) > 0 <J)*(t) = 1 0 < t < Tl 

y (Tl) = o 

Requires a 1 b 1 y 2 (0) < S - (b 2 x 2 (0)) 2 
Tl < T 



S5 



fx, (TS1 ) = 0 



x (Tl) >>0 
2 



<J>* (t ) = 



0 < t < TS1 



0 TS1 < t < Tl 



y (Tl) = o 

Requires S 2 - (b 2 x 2 (0)) 2 _ a j b i y 2 (0) < s : 
+ (R-l) (b 2 x 2 (0)) 2 



Tl < T 



S6 



S7 



x x (Tl) > 0 



x 2 (T1) = 0 



y(Tl) = 0 
x 2 (TS1) = 0 
x 2 (T1) = 0 
y (Tl) > o 



No extremals lead to this state 



$*(t) = 



fl 0 < t < TS1 

0 TS1 < t < Tl 
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Requires a 1 b i y 2 (0) > s 2 + (R-l ) (b 2 x ( 0 ) ) 2 



S8 



Case II. 



SI 



TS1 = 



T1 1 T 
Xj (T1 ) = 0 

•x 2 (T1) =0 No extremals lead to this state. 

,y(Tl) > 0 



ajP < a 2 q 

'xj(T) > 0 

x 2 (T) > 0 
[y (T) > o 

where tSI = 



«>*(t) = 



0 0 <t< T if T S1 > T 



0 < t < T - tSI 



0 T - xSl < t 1 T if tSI < T 



/a 2 b 2 



In 



/? 



Z + /Z 2 + y 2 ~ 1 



1 + Y 



Requires T < TSI where 



■■ tanh 

/ a 2 b 2 



/a^ y ( 0 ) 
s/R~ 



if a x b 1 y ( 0 ) 2 



< R{ s 2 -(b lXl (0)) 2 } 



~ — In (y) if R{ s 2 - 



/a z b 2 



/a 2 b 2 



In 



b 2 x 2 (0) 



— — In { -y> if Rs 2 < 



/a 2 b 2 



(b^CO)) 2 }! a t b x y 2 ( 0 )< Rs 2 
if aj b x y 2 ( 0 ) = Rs 2 
a x bj y 2 (0) 
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Y = 



t^XjCO) /R - /a 1 b 1 y 2 (0) - R{"s 2 - (b^CO)) 2 } 
s/R - /a t b ~ y ( 0 ) 



S2 



1 0 < t < TS2 

0 TS2 < t < T1 



'x (TS2) > 0 
x 2 (T) > 0 4>*(t) = 

y(T) > o 

Requires a 1 b i y 2 (0) > s 2 - (b 2 x 2 (0)) 2 and 
rSl + TS2 < T where 



TS2 = 



v ^7 



In { y } if a 1 b 1 y 2 (0) > s : 



In 



/a i b i 



b 2 x 2 ( 0 ) 



► a 1 b 1 y 2 (0) = s ' 



1 f _• i _„2 /r\\ ^ „ 2 



/a i b i 



In {-y) if a 1 b 1 y 2 (0) < 



Y = 



/a j b j y 2 ( 0 ) - s 2 + (b 2 x 2 (0)) 2 - b 2 x 2 (0) 
^ajb l y ( 0 ) - s 



fv 



S3 



x j (TS3 ) > 0 

x 2 (T) = 0 = 

y(T) > o 



'1 0 < t < TS3 

0 TS3 < t < T 



Requires TS1 < T and T - fSl < TS3 where TS3 is a 
root of 



0 = ^-(1 + — )[s cosh (Yj) - /a^'y(O) sinh (Yj)] 

/r 
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+ ^(l - -%)[s cosh (y 2 ) - /iTb y(0) sinh (y )] 
^ /R 



- s cosh ( /a j b : TS3) 



+ /ajbj y ( 0 ) sinh ( /a [ b [ TS3 ) + b 2 x 2 (0) 



Y x = /a7b7 (T - TS3) + TS3 



Y 2 = * / a 2 b 2 (T - TS3) - /a 1 b 1 TS3 



S4 



Xj (T1 ) > 0 
x 2 (T1) > 0 

k y ( ti) = o 



ts4 = 



/a 2 b 2 



In 



+ / 



Z - 1 



(a.) When TS^ > T1 ^ ( t ) = 0 0 < t < T1 

Requires 

a x bj y 2 (0) < Rs 2 | 1 - p- j 

ajbjy 2 (0) < R { s 2 - (bjXjCO)) 2 } 
a 1 b 1 y 2 (0) < s 2 + (R - l)(b 2 x 2 (0)) 2 



T1 < T 



(b. ) When tS4 < T1 

( 1 . ) When R - /R(R - 1) < 6 < 1 






'l 0 < t < T1 - t S4 

0 T1 - < t < T1 

V. 
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Requires 



2 



a i b i y 2 (0) > s 2 + A(b 2 x 2 (0)) 
aib 1 y 2 (0) < s 2 + B(b 2 x 2 (0)) 2 
a i b i y 2 (0) > Rs 2 | 1 - — | 



T1 < T 



(2.) When 6 = R - /R(R-l) 



<J>* (t ) 



'1 0 < t < TS^4 

0 TS4 < t < T1 



Requires 

a 1 b 1 y 2 (0) = s 2 



b 2 x 2 (0) > Mi(°){/S " 1 } 
T1 < T 



where TS^ is any value less than TS2 



S5 



'x (TS2) = 0 
< x 2 (T1) > 0 
y(Tl) = 0 

V 



<j)-* (t ) = 



0 < t < TS6 
TS6 < t < T1 



Requires 

a 1 b 1 y 2 (0) > s 2 + B(b 2 x 2 (0)) 2 
a l b i y 2 (0) < s 2 + (R-l ) (b 2 x 2 ( 0 ) ) 2 
Ti < t 
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S6 



Xj (Tl) > 0 
■ x 2 (T1) = 0 



ly(Ti) = o 






f 

l 

o 

\ 



0 f t 1 TS6 
TS6 < t 5 Tl 



Requires 

a x b >y 2 < s 2 + A (b 2 x 2 (0)) 2 
a 1 b 1 y 2 (0) > R{ s 2 - (bjXj(O)) 2 } 
a 1 b 1 y 2 (0) < s 2 + (R-l) (b 2 x 2 (0)) 2 
Tl 1 T 



where 






TS6=f 



Zi7b7 



In (y) if a 1 b 1 y 2 (0) > s : 



ln { ^T§sr} lf a i b i y2(0) ' s: 






In (-y) if ajb } y 2 (0) < s 



where 



y(TS6) -/y 2 (TS6) - y 2 (0) + 



y ( o ) - — s— 

/a i b i 

and 

A 2 (o) 

y(TS6) = /— { ( 2R-1 )b 2 x 2 ( 0 ) 



+ 2/s 2 " + R(R-I) (b 2 x 2 (0)J 2 - a 1 b 1 y(0) 2 } 
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S7 



0 < t < TS2 



S8 



'xj(Tl) = 0 

x 2 (T1 ) = 0 cf>*(t) = -I 

k y(Ti) > o 



TS2 < t 1 T1 



Requires 

a jb jy 2 ( 0 ) > s 2 + (R-l) (b 2 x 2 (0)) 2 
T1 < T 
x j (T1 ) = 0 

x 2 (T1) =0 No extremals lead to this state 

,y(Tl) > 0 
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